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ABSTRACT 

The process of awarding the grant for the National 
Center for Research in Vocational Education, as authorized by the 
Carl D. Perkins Vocational Education Act, began in November 1986 with 
the announcement of the grant competition and culminated on January 
4, 1988, with the University of California at Berkeley being selected 
for the 5-year grant over Northern Arizona University and Ohio State 
University (where the National Center had been operating under a 
grant that expired on January 15, 1988). To stimulate competition, 
the Department of Education awarded planning grants to four 
institutions that had applied to enter the competition for the 
National Center (Berkeley, Ohio State, Northern Arizona University, 
and Virginia Polytechnic Institute and State University). The first 
three of these institution eventually completed the grant application 
process. Five field readers were selected by the acting assistant 
secretary to read and score applications on the basis of published 
evaluation criteria. Based on these evaluations, Ohio State received 
the top ranking, followed by Berkeley and Northern Arizona. About a 
month later, the field readers made site visits to the three 
applicant institutions. Ohio State and Northern Arizona provided 
tours of the facilities that would be used for the National Center, 
whe<*eas Berkeley officials did not. Instead, they discussed their 
plans for the National Center facilities that would eventually be 
developed by subcontracting institutions. Afterward, the field 
readers revised their scores. This adjustment resulted in increases 
of 2.4, 13.2, and 9.2 points in the respective scores of Ohio State, 
Berkeley, and Northern Arizona. The assistant secretary then 
recommended that Berkeley receive the grant because it had the 
highest score. (MN) 



United States 

General Accounting Office 

Washington, D.C. 20548 



Human Resources Division 

B-205028 

January 29, 1988 

The Honorable Howard M. Metzenbaum 
Chairman, Subcommittee on Labor 
Conimittee on Labor and Human Resources 
United States Senate 

The Honorable William F. Goodling 
Ranking Minority Member 
Subconunittee on Elementary, Secondary, 

and Vocaf onal Education 
Committee on Education and Labor 
House of Representatives 

This report responds to your December 1987 requests that we review 
the Department of Education's recent grant award procedures used to 
select one of three competing universities as the National Center for 
Research in Vocational Education. In discussions with your offices, we 
agreed to (1) describe and analyze the process used to select the 
awardee and (2) examine issues related to the qualifications of 
nonfederal panelists (field readers) selected to evaluate grant applica- 
tions, each applicant's proposed financial contribution toward the 
Center's establishment, whether the difference between the two top- 
ranked applicants' final scores was statistically significant, and several 
other aspects of the selection process. 

The process to select the National Center, authorized by the Carl D. 
Perkins Vocational Education Act, began in November 1986 with the 
announcement of the grant competition and culminated on January 4, 
1988, with the University of California at Berkeley being selected for 
the 5-year, $30 million grant over Northern Arizona University and Ohio 
State University. (Ohio State formerly operated the National Center 
under a grant that expired on January 15, 1988.) 

This report elaborates on information we gave to your offices and other 
interested congressional staff and Department of Education officials in 
January 4, 1988, brietings. Because of pending litigation, we offer no 
opinion regarding the legality of the grant process. Appendix I describes 
the award process used to select Berkeley, and appendix II discusses our 
objectives, scope, and methodology. 



Background ^^^^ ^' P^^^*"^ vocational Education Act (rublic Law 98-524) stip- 

^ ulates that the National Center be associated with a public or private 
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nonprofit university that is prepared to make a substantial financial 
contribution to the center's establishment. The law also requires that the 
center have a director who is appointed by that university. 

The National Center's primary purpose is to design and conduct 
research and developmental projects and programs. Activities may be 
conducted by the National Center itself or by institutions of higher edu- 
cation and other public agencies acting as subcontractors to it. 

Five nonfederal field readers selected by the Department read and 
scored the three grant applications based on preestablished evaluation 
criteria of varied weights. During site visits to the three applicants, field 
readers, as permitted by Department procedures, adjusted their initial 
scores based on their observations and on discussions with institution 
officials that clarified concerns associated with the applications. The 
scores for each application were averaged and ranked. The assistant 
secretary for vocational and adult education then recommended funding 
the top-ranked application (Berkeley). Negotiations took place and 
Berkeley was subsequently selected as the grantee. 

On January 8, 1988, Ohio State University filed suit against the Depart- 
ment of Education alleging violations of the statutes and regulations 
pertaining to the grant award process. 

The Division of Innovation and Development in the Department of Edu- 
cation's Office of Vocational and Adult Education administers the 
National Center's grant program and oversees the grant competition 
every 5 years. Office procedures to implement each grant competition 
are contained in an Application Review Plan. During the recent grant 
award process, leadership of the Office of Vocational and Adult Educa- 
tion was under (1) an acting assistant secretary from October 1, 1985, to 
February 22, 1987; (2) a second acting assistant secretary from Febru- 
ary 23 to June 19, 1987; and (3) an assistant secretary from June 20, 
1987, to the present. 

Institutions competing for the National Center grant were required by 
published program regulations (34 C.F.R. Part 417) to submit applica- 
tions to the Secretary of Education. A panel of field readers read and 
scored applications that met grant eligibility criteria requiring appli- 
cants to be (1) prepared to make a substantial financial contribution 
toward the Center's establishment and (2) associated with a public or 
private nonprofit university. 
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In accordance with program regulations (34 C.F.R. 417.31), field readers 
scored each application on the basis of seven evaluation criteria of 
varied weights established by the Department: required functions cover- 
ing services and activities (40 points), management (20 points), quality 
of key personnel (15 points), experience in and capacity for conducting 
research in vocational education (10 points), adequacy of resources (5 
points), budget and cost effectiveness (5 points), and external relations 
with individuals at the national, state, and local levels (5 points). In 
addition, the Application Review Plan required site visits to the appli- 
cants. Such site visits were required so that field readers could further 
evaluate applications and subsequently revise scores if appropriate. The 
Department's Grants and Contracts Service standardized' the readers' 
raw scores and ranked the applications based on these standardized 
scores. 

According to Department regulations (34 CRR. 75.217(d)), the secre- 
tary or designee who recommends an application for funding may con- 
sider (1) the information in each application; (2) the ranking of grant 
applications; and (3) any other information relevant to a criterion, prior- 
ity, or other requirement that applies to the selection of applications for 
new grants. If negotiations with the applicant recommended for funding 
are successful, the applicant is formally selected as the grantee. 



The, Perkins Act requires field readers for the National Center grant 
competition to be recognized nationally as experts in vocational educa- 
tion administration and research. To gather information concerning this 
requirement, we (1) reviewed the Department's process for selecting 
field readers and (2) asked the heads of four national organizations 
related to vocational education, a national advisory council on voca- 
tional education appointed by the President, and the congressionally 
\iandated National Assessment of Vocational Education if they were 
familiar with the backgrounds and experiences of the five readers. None 
of the individuals from these organizations had heard of or could attest 
to the qualifications of three of the five readers. 

In a December 30, 1986, memorandum to the then acting assistant secre- 
tary for vocational and adult education, the director of the Division of 



'Standardization is a Matislical procedure that minimizes l lie londcnrv of some readers to score apph- 
catums consistently high or consistently low However, in the case of the National Center s grant 
competition, standardizing the scores did not affect the final rankings 



Selection of Field 
Readers 
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Innovation and Development recommended as field readers 44 individu- 
als who he stated were **experts recognized in the areas of vocational 
administration and research." The names were grouped under the head- 
ings **research,'* **state agency personnel," **community college person- 
nel," and **other names." Each name was accompanied by a brief 
description of the individual's activities in vocational education, such as 
**strong in financing vocational education," **director of a research insti- 
tute," or **knows practical problems i't vocational education." The direc- 
tor also attached two additional lists for the acting assistant secretary's 
consideration, made up of (1) the five persons, who had reviewed the 
National Center's continuation grant application in the previous year 
and (2) the five persons who were scheduled to review the National 
Center planning grant applications on January 12-16, 1987. 

According to one of the staff member-^ who developed the first list of 44 
names, the Application Review Plan contained no written criteria for 
selecting ''individuals recognized nationally as experts in vocational 
education administration and research." Criteria were never developed, 
he said, because it was felt to be difficult to further define this statutory 
requirement. In his opinion, **national experts" would likely be individu- 
als known by various vocational associations, state organizations, and 
the Department for their contributions to vocational education. In addi- 
tion, he said that staff interpreted the law to mean that a field reader 
should be a nationally recognized expert in either administration or 
research but that both disciplines should be represented on the panel. 

The official said that in developing list of potential field readers, he 
and the other staff member identifiea individuals they knew or had 
knowledge of in the field. According to him, they identified some names 
through his personal file of prominent individuals in vocational educa- 
tion or a newly established division file of potential field readers con- 
taining about 20 resume.^. 

In February 1987, the newly appointed acting assistant secretary for 
vocational and adult education selected five readers. Of the 5, 2 were 
from the suggested list of 44 names; 1 was from the list of readers who 
participated in the previous year's review of the National Center's appli- 
cation for continued funding; and 1 was from the list of readers who 
participated in the review of planning grant applications for the 
National Center. The fifth reader was not on the lists of potential reaa- 
ers developed by the program officp and sent to the acting assistant 
secretary. 
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The 2 individuals selected from the list of 44 names had been listed 
under the heading **state agency personnel/' One was described as an 
^^experienced research coordinating unit director/' and the other as an 
**experienced administrator of both secondary and post-serondary voca- 
tional education programs/' These two individuals were also the or.as 
whose names were recognized by at least some of the six vocational edu- 
cation organizations and two of the three representatives of the compet- 
ing institutions (see p. 6). 

In making his selections, the former acting assistant secretary consid- 
ered the lists of names submitted to his predecessor. He said that the 
individual he selected, who was not on any of the lists, was a friend and 
former colleague whom he selected because of the person's experience in 
postsecondary education and vocational education research. The acting 
assistant secretary also stated that it was difficult selecting field readers 
because many qualified individuals had previous contact with Ohio 
State University's National Center for Resei^i^ch in Vocational Education. 



Table 1: Vocational Education 
Organizations' Familiarity With Panelists 



Organ ization 

American Association of Community and 
Junior Colleges 

Amencan Vocational Association 

Amencan Vocational Education Research 
Association 

National Assessment of Vocational Education 

NatiOi,al Association of State Directcs of 
Vocational Education 

.Niational Council on Vocational Education 



Panelist 

1 2 3 4 5 

No No No No No 
Yes No No No No 



No No No No No 

Yes Yes No No No 

Yes Yes No No No 

Yfes No No No No 



A "Yes" indicates that the organization was familiar with the panelist 



1.3 m appendix I. 



Because the Department had no criteria to select "nationally recognized 
experts/' we surveyed the heads of vocational education organizations 
to determine whether they had heard of or could attest to the qualifica- 
tions of the panelists. According to the program official who assisted us 
in identifying these organizations, they are **key'' vocational education 
related groups. As shown in table 1, four of the six organization heads 
we contacted indicated familiarity with one reader and two indicated 
familiarity with another.^ None of the organization heads had heard of 
or could attest to the qualifications of the other three readers. We 

^To ensure reader anonymity, panelist numbers in table 1 do not coincide with those m tables I 2 and 
1.3 m appendix I. 
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attexTipted to verify positive responses from the organization heads by 
requiring respondents to substantiate their answers with evidence of 
specific knowledge of the panelists' qualifications. 

We obtained similar results from our telephone interviews with the pro- 
posed center directors for each of the three applicants. For example, the 
two readei-s whose backgrounds and experiences were fanfiiliar to one or 
more of the heads of the vocational education organizations were also 
familiar to two of the three officials representing the applicants. In addi- 
tion, two of the three readers v/hose backgrounds and experiences were 
not recognized by any of the organization heads we contacted were also 
not recognized by any of the three applicants. 



The Perkins Act contains legal requirements that a grantee must meet to 
obtain the 5-year National Center grant. These requirements relate to a 
grantee's (1) preparing to make a substantial financial contribution 
toward the National Center's establishment, (2) being an organization 
associated with a public or private nonprofit institution, and (3) 
appointing a director for the National Center. 

Regarding the issue of substantial financial contribution, Berkeley pro- 
posed a first year contribution of $686,000 compared to $ 1.6 million 
proposed by Ohio State University and $2.1 million^ proposed by North- 
em Arizona University. The Application Review Plan contained no crite- 
ria to define what constitutes a substantial financial contribution, and 
the division, which screened grant applications, approved the eligibility 
of all three. Without such criteria, a program official who helped screen 
applications toid us that it was agreed that the staff would use its pro- 
fessional judgment and experience in screening the applications. He said 
that although Berkeley's proposed contribution of $686,000 was less 
than those of the other two applicants, the amount was considered **sub- 
stantial" because it represented 10 percent of the $6 million annual 
grant amount to be awarded. 

The same official also told us chat Berkeley's proposal to affiliate with 
other universities to provide grant services was acceptable under the 
law because it permits subcontracting with **other public agencies and 
public or private institutions of higher education." However, according 



^According to a Jepartment grants offiaal, if Northern Arizona University had been recommended 
for funding, its proposed contribution would not have been allowed because it represented the value 
of real property rather than allowable expenses such as staff time and overhead rates. 



Department 
Consideration of Grant 
Eligibility Factors 
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to a gr^^nts official responsible for negotiating budgetary matters with 
recommended grant recipients, Berkeley initially also proposed to share 
fiscal responsibility with the other universities with which it proposed 
to .afiliate. He noted that it was the Department's position that the 
applicant, regardless of its subcontracting or affiliation arrangements, 
must assume all f ''cal responsibility for the grant. This was a major dis- 
cussion point in ohe Department's grant negotiations with Berkeley, but 
was satisfactorily resolved when Berkeley agreed to be solely responsi- 
ble for its contribution and consequently increased its proposed contri- 
bution from $93,000 to $603,000. 

After eligibility was granted, the program official who served as chair- 
man of the grant review panel expressed concern with Berkeley's propo- 
sal to have two codirectors. The Perkins Act provides that the **National 
Center shall have a Director." In response to his concern that Berkeley's 
proposal of two codirectors might be inconsistent with the act, he was 
instructed by the division director to inform field readers during the 
application review period in iate August 1987 that (1) all applications 
were eligible for review and (2) the issue relating to Berkeley's director- 
ship should not be considered in evaluating applications. On September 
1, 1987, the division director sought a legal opinion on this matter from 
the Department's Office of General Counsel. 

In late September, a Department attorney advised the program official 
by telephone that Berkeley's proposal of two codirectors was a **techni- 
cal" matter, which did not affect grant eligibility. In accordance with the 
attorney's instructions, the program official advised field readers not to 
raise the issue of codirectors with Berkeley officials during the site visit. 
An October 22, 1987, Department legal opinion stated that: 

*\ . . It would be inappropriate to regard the proposing of co-directors in the applica- 
t:'^'^ as affecting the overall eligibility of the applicant. This would be an overly 
strict reading of the statute; we could find no legislative history to support such a 
reading. Even if the statute were read to bar naming two co-oirectors, the appli- 
cant's proposal to use co-directors would constitute a technical deficiency, which 
could be rectified at a later time in the negotiation stage/' 

After grant negotiations, Berkeley agreed '.o name an executive director 
and an associate director. 
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Before making site visits to the applicant institutions during which read- 
ers could revise initial scores for each applicant, four of the five field 
readers knew the rank order of the applications based on readers' 
assigned application scores. We found no evidence to suggest that the 
Department provided this information to influence the process or that 
field readers used this information to unfairly influence other readers to 
change their scores. However, withholding information on panelists' ini- 
tial total scores could reduce the possibility of any panelist manipulating 
the scoring system by arbitrarily increasing his or her score to offset a 
lower score by another panelist. 

Accorc"ing to the five readers and program and grants officials who 
attended the review session, each reader's scores for each application, 
by the seven criteria, as well as total scores, were recorded on a black- 
board in the meeting room on the last day of the review period in 
August 1987. According to program officials, scores were displayed in 
this manner to give reviewers, who assigned the highest and lowest 
scores on each criterion, an opportunity to explain the perceived 
strengths and weaknesses of the applications, after which field readers 
could change their scores if they desired. The program official who 
chaired the field reader panel told us that no field reader cha d his or 
her score at that time. 

Four of the five field readers told us that, using the raw scores dis- 
played, they were able to compute the rank order of the applications. 
Each of the four readers recalled that Ohio State was ranked first; 
Berkeley, second; and Northern Arizona, third. The other field reader 
stated that although total scores were displayed, he did not compute the 
rank order or otherwise learn of it. 

Program and grants officials said that discussions cf field readers' low 
and high scores are a routine part of application reviews. However, they 
acknowledged that most grant competitions have more gpplicants and 
more panels of readers, which would make it more difficult for readers 
to compute the overall rank order of applicants. They agreed that loiow- 
ing the applicants' rank order hef ore the completion of the review could 
have allowed the misuse of this information. 



Rank Order of 
Applicants Known 
Before Completion of 
Scoring Process 
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Statistical Significance 
of Difference in Final 
Scores 



After our January 4, 1988, briefing, your offices asked us to determine 
if the difference in the final scores between Berkeley, ihe top-ranked 
applicant, and Ohio State, the second-ranked applicant, was statistically 
significant. In our opinion, such a determination can be made only if che 
readers who evaluate grant proposals are considered to be a sample of 
the universe of potential readers. Because there were only five panelists 
to review these proposals "J^e do not believe that such a test of these 
scores is appropriate considering the limited sample size. We estimate 
that about 30 to 60 readers would be needed to conclusively determine 
whether there is any statistically significant difference between the 
Berkeley and Ohio State scores. 



As requested by your offices, we did not obtain written agency com- 
ments on a draft of this report. However, we discussed its contents with 
Department officials and incorporated their comments where appropri- 
ate. We also incorporated pertinent changes suggested in a January 26, 
1988, letter from the assistant secretary for vocational and adult educa- 
tion to our office concerning issues discussed during our January 4 
briefing. Because these matters pertained to our briefing rather than our 
written report, we did not include a copy of these comments in our final 
report. 

Also, as requested by your offices, we plan no further distribution of 
this report until 7 days after issuance unless its contents are publicly 
announced earlier. At that time, we will send copies of this report to the 
Director, Office of Management and Budget; the Secretary of Education; 
the three grant applicants; and other interested congressional parties. 
Copies will be made available to others on request. 




William J. Gamer 
Associate Director 
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National Research Center Grant Award Process 



To stimulate competition, the Department provided planning grants in 
late February and early March 1987 to institutions interested in apply- 
ing for the National Center grant. Under a competitive process initiate ', 
in August 1986, four institutions applied for and were ultimately 
awarded $25,000 each to develop applications for the National Center 
grant competition. They were Berkeley, Northern Arizona University, 
Ohio State University, and Virginia Polytechnic Institute and State 
University. 

On November 28, 1986, the Department published a notice in the Fed- 
eral Register inviting grant applications for the National Center. Appli- 
cations were due on August 14, 1987. Three of the four institutions that 
received planning grants submitted applies "ons: Berkeley, Northern 
Arizona, and Ohio State.* There were no other applicants. 

Upon receiving the applications, officials in the Division of Innovation 
and Development reviewed them and determined that all were eligible 
for the competition. According to eligibility criteria stated in published 
program regulations and the Office of Vocational and Adult Education's 
Application Review Plan, an applicant was considered eligible if it was 
(1) associated with a public university or private, nonprofit university 
and (2) prepared to make a substantial financial contribution toward the 
Center's establishment. The screening criteria in the regulations and the 
Application Review Plan did not contain reference to the legislative 
requirement that "a director'' be designated. 

The five field readers, selected by the acting assistant secretary, con- 
vened for 5 days in Washington, D.C., on August 24, 1987, to read and 
score ^plications. These readers scored the applications on the basis of 
published evaluation criteria. About a month later, from September 28 
through October 1, 1987, the field readers made site visits to the three 
applicant institutions. These visits were miJe so that the readers could 
obtain more information about each applicant's grant application, facili- 
ties, and policies and subsequently revise scores if appropriate. 

A September 4, 1987, letter to each applicant stated that appropriate 
activities for the site visit inciuded formal presentations expanding, sup- 
plementing, or summarizing information in the application; tours of the 
proposed facilities for the National Center; and slide presentations or 
showing of blueprints of additional off-site facilities. 



* Virginia Polytechnic Institute and State University was one of five proposed subcontracting institu- 
tions in Berkeley's grant application. 
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The Department's letter to the applicants also stated that **identical pro- 
cedures will be followed for each of the site visits to ' he other appli- 
cants'' and **the same agenda will also be followed at each site/' The 
agenda, a copy of which was attached to each letter, specified 2 hours 
for a presentation and facilities review, 2-1/2 hours for a question-and- 
answer session, and 1-1/2 hours for panelists to write comments justify- 
ing any revised scores. 

During their designated 2 hours for a presentation and facilities review, 
Ohio State and Northern Arizona provided tours of their proposed facili- 
ties for the National Center, while Berkeley did not. According to the 
Department program official who chaired the field reader panel and 
attended the site visits, Berkeley officials discussed their facilities to be 
used for the National Center during a 2-hour presentation and slide 
show of the campuses of its proposed subcontracting institutions. This 
official said the slides did not show the particular facilities to be used 
for the National Center. 

According to program officials, there was a 1-month time lag between 
the application review period and the site visits to resolve logistical 
problems. In telephone interviews with field readers, all stated that dur- 
ing the month between the application review and site visits, they did 
not receive any new informa '>n about the applicants or the 
applications. 

In accordance with the Application Review Plan, field readers were 
allowed, after each site visit, to ac^just any scores previously given dur- 
ing the application review on the basis of information obtained from the 
visit. Table LI shows the extent to which overall average scores 
changed as a result of the site visits. Tables 1.2 and 1.3 show the extent 
to which readers adjusted their scores for the two top-ranked proposals. 

Tab!? l/i: Field Readers' Scoring 
Adjustments 





Average 








scores 


Average 




Applicant 


before site 


scores after 


Total 


visits 


site visits 


adjustments 


Ohio State 


81.2 


83 6 


+ 24 


Berkeley 


73 6 


86.8 


+13.2 



Northern Arizona 56.6 65 8 +92 
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As shown in table I.l, at the end of the panel meeting in Washington, 
D.C., Ohio State's average raw score was 7.6 points higher than Berke- 
ley's, but after readers ac^usted their scores during the site visits, 
Berkeley's average raw score was 3.2 points higher than Ohio State's. 



Table 1.2: Panelists' Raw Scores Before and After Site Visit— Berkeley 





Maximum 






Panelist 






Criteria 


points 


1 


2 


3 


4 


5 


Required functions 


40 


25/32 


30/35 


28/30 


34/38 


34/36 


Management 


20 


13/17 


10/16 


14/16 


12/18 


10/16 


Personnel 


15 


12/14 


10/12 


10/12 


10/15 


14/14 


Vocational education experience 


10 


9/9 


7/8 


8/8 


10/10 


10/10 


Resources 


5 


5/5 


5/5 


4/5 


2/5 


5/5 


Budget 


5 


5/5 


5/5 


3/3 


5/5 


3/4 


External relations 


5 


4/4 


3/5 


3/3 


4/5 


2/4 


Total 


100 


73/86 


70/86 


70/77 


77/96 


78/89 



Table 1.3: Panelists' Raw Scores Before and After Site Visit— Ohio State University 





Maximum 






Panelist 






Criteria 


points 


1 


2 


3 


4 


5 


Required functions 


40 


36/36 


26/26 


30/33 


36/36 


35/35 


Management 


20 


17/17 


13/17 


16/17 


17/17 


16/18 


Personnel 


15 


11/11 


5/5 


11/13 


13/13 


12/10 


Vocational education expenence 


10 


10/10 


6/6 


10/10 


10/10 


9/9 


Re«:ources 


5 


5/5 


5/5 


5/5 


5/5 


5/5 


Budget 


5 


5/5 


5/5 


3/3 


5/5 


5/5 


External relations 


5 


4/4 


2/4 


4/4 


5/5 


4/4 


Total 


100 


88/88 


62/68 


79/85 


91/91 


86/86 



The Department's Grants and Contracts Service standardized the read- 
ers' raw scores and ranked the applications based on the standardized 
scores. Applicants' final rankings were provided to the assistant secre- 
tary on October 14, 1987. Later that day, she recommended the funding 
of Berkeley, the top-ranked applicant, based on the final rankings and 
the program staff's recommendation. 

The assistant secretary said that she recommended funding Berkeley to 
the Grants and Contracts Service on the basis of its relative ranking 



C D I Page 14 GAO/HRD-88^6 Vocational Education Research Center 



Appendix I 

National Research Center Grant 
Award Process 



with other institutions and did not realize other factors could be consid- 
ered. Federal regulations do state, however, that the assistant secretary 
may also consider the following in making grant award selections: 

• The information in each application. 

• Any other information relevant to a criterion, priority, or other require- 
ment that applies to the selection of applications for new grants. 

The assistant secretary told us that based on separate discussions she 
had v/ith program and grants office staff regarding her responsibilities 
in the award selection process, she believed it was the normal procedure 
to reconmiend that the top-ranked application be funded regardless of 
the point difference. She said that she did not read any of the applica- 
tions before making her decision. In fact, she believed if she had consid- 
ered factors other than application scores in making her decision, she 
would have been criticized for having a conflict of interest because she 
had received a doctorate in education from Berkeley in 1985. In later 
discussions, Department officials told us that selections of applicants 
that deviate from the rank order must be justified in writing and are 
rarely made. They said such deviations occur primarily in cases where 
there is a compelling reason not to fund a top-ranked applicant; for 
example, if a top-ranked applicant had previously demonstrated inade- 
quate grant performance. 

After the assistant secretary's decision, program officials and officials 
of the Department's Grants and Contracts Service conducted negotia- 
tions with Berkeley. During the negotiations, technical and budgetary 
issues of Berkeley's grant application were resolved. Negotiations were 
concluded on December 18, 1987, and the Department formally awarded 
the grant to Berkeley on January 4, 1988. We were told that if negotia- 
tions with Berkeley had been unsuccessful, the second-ranked applicant 
would have been recommended for funding. 
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In a December 1, 1987, letter from the Chairman, Subcommittee on 
Labor, Senate Committee on Labor and Human Resources, and a Decem- 
ber 2, 1987, letter from the Ranking Minority Member, Subconmiittee on 
Elementary, Secondary, and Vocational Education, House Committee on 
Education and Labor, we were asked to review the Department of Edu- 
cation's recent grant award process for the National Center for Research 
in Vocational Education. In discussions with the requesters' offices, we 
agreed to analyze and describe the grant award process and examine, 
amjng other things, (1) the field reader selection process; (2) each appli- 
cant's proposed financial contribution toward the establishment of a 
National Center; and (3) the statistical significance, if any, of differences 
in final scores between the two top-ranked applicants. As agreed, we did 
not evaluate the substance of the grant applications or the field readers' 
written comments on them. 

We reviewed applicable legislation and regulations; the Office of Voca- 
tional and Adult Education's approved Application Review Plan, which 
specified procedures for conducting the National Center's grant competi- 
tion; and other Department documentation related to the grant competi- 
tion. We also interviewed Department officials, including the assistant 
secretary for vocational and adult education, program officials, and offi- 
cials in the Grants and Contracts Service. 

To obtain information on the grant award process, we conducted struc- 
tured telephone discussions with the five field readers. In addition, we 
contacted the heads of six vocational education organizations to deter- 
mine the extent to which they had heard of or could attest to the qualifi- 
cations of the readers. These organizations were: the American 
Vocational Association, the American Vocational Education Research 
Association (an arm of the American Vocational Association), the 
National Assessment of Vocational Education (a congressionally man- 
dated study group located in the Department of Education), the National 
Association of State Directors of Vocational Education, the National 
Council on Vocational Education (a national advisory council appointed 
by the president), and the American Association of Community and Jun- 
ior Colleges. According to the program official who assisted ui ' \ identi- 
fying these organizations, they are **key" vocational education related 
groups. We also talked with officials representing each of the three 
applicants to obtain information on site visits and discuss field readers' 
qualifications. 

As directed by the requesters* offices, we did not obtain official agency 
comments on this report. However, we discussed its contents with 
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Department officials and incorporated their comments where appropri- 
ate. Our audit work was conducted during December 1987. Our review 
was conducted in accordance with generally accepted government audit- 
ing standards. 
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Requests for copies of gao reports should be sent to: 

U.S. General Accounting Office 
Post Office Box 6015 
Gaithersburg, Marylana 20877 

Telephone 202-275-6241 

The first five copies of each report are free. Additional copies are 
$2.00 each. 

There is a 25% discount on orders for 100 or more copies mailed to a 
single address. 

Orders must be prepaid by cash or by check or money order made out to 
the Superintendent of Documents. 
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